Towards a Reliable Comparison and Evaluation of Network Intrusion Detection Systems Based on Machine Learning Approaches

Presently, we are living in a hyper-connected world where millions of heterogeneous devices are continuously sharing information in different application contexts for wellness, improving communications, digital businesses, etc. However, the bigger the number of devices and connections are, the higher the risk of security threats in this scenario. To counteract against malicious behaviours and preserve essential security services, Network Intrusion Detection Systems (NIDSs) are the most widely used defence line in communications networks. Nevertheless, there is no standard methodology to evaluate and fairly compare NIDSs. Most of the proposals elude mentioning crucial steps regarding NIDSs validation that make their comparison hard or even impossible. This work firstly includes a comprehensive study of recent NIDSs based on machine learning approaches, concluding that almost all of them do not accomplish with what authors of this paper consider mandatory steps for a reliable comparison and evaluation of NIDSs. Secondly, a structured methodology is proposed and assessed on the UGR'16 dataset to test its suitability for addressing network attack detection problems. The guideline and steps recommended will definitively help the research community to fairly assess NIDSs, although the definitive framework is not a trivial task and, therefore, some extra effort should still be made to improve its understandability and usability further

Multivariate Statistical Network Monitoring-Sensor: An effective tool for real-time monitoring and anomaly detection in complex networks and systems

Technology evolves quickly. Low-cost and ready-to-connect devices are designed to provide new services and applications. Smart grids or smart health care systems are some examples of these applications. In this totally connected scenario, some security issues arise due to the large number of devices and communications. In this way, new solutions for monitoring and detecting security events are needed to address new challenges brought about by this scenario, among others, the real-time requirement allowing quick security event detection and, consequently, quick response to attacks. In this sense, Intrusion Detection Systems are widely used though their evaluation often relies on the use of predefined network datasets that limit their application in real environments. In this work, a real-time and ready-to-use tool for monitoring and detecting security events is introduced. The Multivariate Statistical Network Monitoring-Sensor is based on the Multivariate Statistical Network Monitoring methodology and provides an alternative way for evaluating Multivariate Statistical Network Monitoring-based Intrusion Detection System solutions. Experimental results based on the detection of well-known attacks in hierarchical network systems prove the suitability of this tool for complex scenarios, such as those found in smart cities or Internet of Things ecosystem

Improving the Reliability of Network Intrusion Detection Systems Through Dataset Integration

This work presents Reliable-NIDS (R-NIDS), a novel methodology for Machine Learning (ML) based Network Intrusion Detection Systems (NIDSs) that allows ML models to work on integrated datasets, empowering the learning process with diverse information from different datasets. We also propose a new dataset, called UNK22. It is built from three of the most well-known network datasets (UGR'16, USNW-NB15 and NLS-KDD), each one gathered from its own network environment, with different features and classes, by using a data aggregation approach present in R-NIDS. Therefore, R-NIDS targets the design of more robust models that generalize better than traditional approaches. Following R-NIDS, in this work we propose to build two well-known ML models for reliable predictions thanks to the meaningful information integrated in UNK22. The results show how these models benefit from the proposed approach, being able to generalize better when using UNK22 in the training process, in comparison to individually using the datasets composing it. Furthermore, these results are carefully analyzed with statistical tools that provide high confidence on our conclusions. Finally, the proposed solution is feasible to be deployed in network production environments, not usually taken into account in the literature.16 página

Protocolo para la notificación y alerta de eventos de seguridad en redes ad-hoc

Las líneas de defensa de seguridad tradicionales para proteger un sistema dado son prevención, detección y respuesta. A pesar de que sobre el papel dichos módulos deben inter-operar a fin de conseguir una seguridad integral, por lo general se plantean y adoptan como soluciones independientes. El presente trabajo aborda el estudio y desarrollo de un protocolo de notificación y alerta de eventos de seguridad cuyo fin principal es servir de interfaz entre los módulos de detección y respuesta. Ideado específicamente para redes ad-hoc, su uso posibilita poner en conocimiento de los elementos constitutivos del entorno monitorizado la ocurrencia de un cierto comportamiento malicioso detectado. Este conocimiento será clave para la ejecución posterior de los mecanismos de respuesta oportunos. También susceptible de ser usada para la distribución de información en procesos de detección/respuesta colaborativos, nuestra propuesta viene a cubrir una carencia manifiesta en el campo objeto de estudio.Este trabajo ha sido parcialmente financiado por el MICINN a través del proyecto TEC2011-22579 y por el MECD a través de la beca del programa de “Formación de Profesorado Universitario” (FPU, Ref.: AP2009-2926)

Optimal Relay Placement in Multi-hop Wireless Networks

Relay node placement in wireless environments is a research topic recurrently studied in the specialized literature. A variety of network performance goals, such as coverage, data rate and network lifetime, are considered as criteria to lead the placement of the nodes. In this work, a new relay placement approach to maximize network connectivity in a multi-hop wireless network is presented. Here, connectivity is defined as a combination of inter-node reachability and network throughput. The nodes are placed following a two-step procedure: (i) initial distribution, and (ii) solution selection. Additionally, a third stage for placement optimization is optionally proposed to maximize throughput. This tries to be a general approach for placement, and several initialization, selection and optimization algorithms can be used in each of the steps. For experimentation purposes, a leave-one-out selection procedure and a PSO related optimization algorithm are employed and evaluated for second and third stages, respectively. Other node placement solutions available in the literature are compared with the proposed one in realistic simulated scenarios. The results obtained through the properly devised experiments show the improvements achieved by the proposed approach

A Model of Data Forwarding in MANETs for Lightweight Detection of Malicious Packet Dropping

This work introduces a model of data forwarding in MANETs which is used for recognizing malicious packet dropping behaviors. First, different legitimate packet discard situations are modeled, such as those generated by collisions, channel errors or mobility related droppings. Second, we propose an anomaly-based IDS system based on an enhanced windowing method to carry out the collection and analysis of selected crosslayer features. Third, a real deployment of the IDS is also considered by suggesting a methodology for the collection of the selected features in a distributed manner. We evaluate our proposal in a simulation framework and the experimental results show a considerable enhancement in detection results when compared with other approaches in the literature. For instance, our scheme shows a 22% improvement in terms of true positives rate and a remarkable 83% improvement in terms of false positives rate when compared to previous well-known statistical solutions. Finally, it is notable the simplicity and lightweightness of the proposal

UGR’16: Un nuevo conjunto de datos para la evaluación de IDS de red

[ES] La evaluación de algoritmos y técnicas para implementar sistemas de detección de intrusiones depende en gran medida de la existencia de conjuntos de datos (dataset) bien diseñados. En los últimos años, se ha realizado un gran esfuerzo para construir estos datasets. En este artículo se presenta un nuevo dataset que se construye con tráfico real y ataques actualizados. La principal ventaja de este conjunto de datos sobre los anteriores es su utilidad para la evaluación de IDSs que consideran la evolución a largo plazo y la periodicidad del tráfico. También permite entrenar y evaluar modelos que consideren las diferencias entre día/noche o entre días laborables/fines de semana.Este trabajo ha sido parcialmente financiado por el Gobierno Espanol-MINECO (Ministerio de Economía y Competitividad) y fondos FEDER, a traves del proyecto TIN2014-60346-RMaciá-Fernández, G.; Camacho, J.; Magán-Carrión, R.; Fuentes-García, M.; García-Teodoro, P.; Theron, R. (2018). UGR’16: Un nuevo conjunto de datos para la evaluación de IDS de red. En XIII Jornadas de Ingeniería telemática (JITEL 2017). Libro de actas. Editorial Universitat Politècnica de València. 71-78. https://doi.org/10.4995/JITEL2017.2017.6520OCS717

SIMAGRO: Un prototipo para la detección de anomalías en entornos IoT para el sector agroalimentario

El sector primario es uno de los m ́as relevantes en Andalucía. Una de las ́areas m ́as importantes dentro de este sector es la agricultura, destacando la producción de aceituna, frutas y hortalizas tropicales, además de los cultivos ecológicos (estos últimos suponen la mitad del total en España). Tras los a ̃nos que se han sucedido de crisis, uno de los pilares fundamentales para que se reactive este sector es la optimización de las técnicas de cultivo, lo que implica la necesidad de una transformación digital profunda. Por esta razón, la sensorización de plantaciones agrarias y la implantación del IoT (del inglés, Internet of Things) como mecanismo de monitorización de los cultivos supone un gran avance para las entidades que lo están implantando. ÉGIDA es la primera Red de Excelencia Cervera para la privacidad y la seguridad de los datos. Uno de los objetivos de esta Red es concienciar sobre la necesidad de llevar a cabo una digitalización segura. En este sentido, existe una alta implicación con la securizaci ́on activa de los entornos IoT, concretamente en el sector agroalimentario. En este contexto, y fruto de la colaboración activa entre la Universidad de Granada (UGR) y Fidesol, se ha llevado a cabo el desarrollo un prototipo para la detecci ́on de anomalías en entornos IoT para el sector agroalimentario. Este prototipo aplica por primera vez el sensor MSNM (MSNM-S) en un escenario IoT. El objetivo de este artículo es doble: por un lado, probar el funcionamiento de Atenea Lab y, por otro, presentar los resultados de la evaluación de este prototipo y resolver las siguientes cuestiones: i) ¿Es aplicable MSNM jerárquico a entornos IoT? y ii) ¿Cómo afecta la configuración de MSNM-S a entornos IoT? Además, se pretende identificar posibles puntos de mejora para continuar evolucionando tanto el prototipo obtenido para IoT como el sensor de MSNMEste trabajo está financiado en parte por las Ayudas Cervera para Centros Tecnológicos del Centro Españool para el Desarrollo de Tecnolog ́ıa Industrial (CDTI) en el marco del proyecto EGIDA (CER-20191012) y por el Ministerio de Ciencia e Innovación (MICIN) MICIN/AEI/10.13039/501100011033, bajo los proyectos PID2020-113462RB-I00 y PID2020- 114495RB-I00, así como los proyectos PPJIA2022-51 y PPJIA2022-52 de ayudas del plan propio de la UGR

Multivariate Statistical Approach for Anomaly Detection and Lost Data Recovery in Wireless Sensor Networks

Data loss due to integrity attacks or malfunction constitutes a principal concern in wireless sensor networks (WSNs). The present paper introduces a novel data loss/modification detection and recovery scheme in this context. Both elements, detection and data recovery, rely on a multivariate statistical analysis approach that exploits spatial density, a common feature in network environments such as WSNs. To evaluate the proposal, we consider WSN scenarios based on temperature sensors, both simulated and real. Furthermore, we consider three different routing algorithms, showing the strong interplay among (a) the routing strategy, (b) the negative effect of data loss on the network performance, and (c) the data recovering capability of the approach. We also introduce a novel data arrangement method to exploit the spatial correlation among the sensors in a more efficient manner. In this data arrangement, we only consider the nearest nodes to a given affected sensor, improving the data recovery performance up to 99%. According to the results, the proposed mechanisms based on multivariate techniques improve the robustness of WSNs against data loss.This work has been partially supported by Spanish MICINN (Ministerio de Ciencia e Innovación) through Project TEC2011-22579, by Spanish MINECO (Ministerio de Economía y Competitividad) through Project TIN2014-60346-R, and the FPU P6A grants program of the University of Granada